Inside Netflix’s bet on advanced video encoding

🌈 Abstract

The article discusses how Netflix's senior encoding technology director, Anne Aaron, has spent the past 13 years optimizing the way Netflix encodes its movies and TV shows. It covers the evolution of Netflix's encoding strategies, from using a one-size-fits-all approach to adopting a per-title encoding method and further refining it to a per-shot encoding approach. The article also delves into Netflix's contributions to the development of new video codecs like AV1 and its successor, as well as the company's efforts to create custom video assets for testing and evaluating encoding algorithms. Additionally, the article explores the challenges Netflix faces as it ventures into new areas like live streaming and cloud gaming, which require a different approach to encoding compared to on-demand video streaming.

🙋 Q&A

[01] Encoding Strategies at Netflix

1. What were the key changes in Netflix's encoding strategies over time?

• Initially, Netflix used a one-size-fits-all approach, encoding each video with a predefined list of bitrates and resolutions.
• In 2015, Netflix began re-encoding its entire catalog with settings fine-tuned per title, allowing for 20% bandwidth savings on average.
• In 2018, Netflix started breaking down videos by shots and applying different encoding settings to each individual segment to better handle visually complex scenes.
• Netflix has also developed custom video assets like the 4K HDR short film "Meridian" to test and evaluate encoding algorithms.

2. How did Netflix's per-title and per-shot encoding approaches improve streaming quality and efficiency?

• The per-title encoding approach allowed Netflix to use lower bitrates for simpler visual content like animation, while maintaining quality for more complex live-action titles.
• The per-shot encoding approach enabled Netflix to apply optimal encoding settings for each individual segment of a video, handling visually challenging scenes more effectively.
• These encoding strategies resulted in significant bandwidth savings, allowing Netflix to deliver better-looking streams over slower connections, especially important as the company expanded globally.

[02] Contributions to Video Codec Development

1. What is Netflix's involvement in the development of new video codecs?

• Netflix is a founding member of the Alliance for Open Media, which has spearheaded the development of the open, royalty-free AV1 video codec.
• Netflix has encoded about two-thirds of its catalog in AV1 and is seeing "close to 30% bit rate reduction with the same quality compared to AV1" for its successor codec.
• Netflix has also created custom video assets like the "Meridian" short film to help the industry test and evaluate encoding algorithms and codecs.

2. How has Netflix's involvement in codec development benefited the streaming industry?

• Netflix's contributions to the development of AV1 and its successor codec have helped advance the state of video compression technology, enabling more efficient streaming.
• The release of the "Meridian" short film under a Creative Commons license has provided the industry with a more representative set of video content for testing and evaluating codecs, beyond the limited freely available clips previously used.

[03] Challenges in Live Streaming and Cloud Gaming

1. What are the key challenges Netflix faces in transitioning to live streaming and cloud gaming?

• Live streaming requires a different approach to encoding compared to on-demand video, as the content is unpredictable and the latency requirements are more stringent.
• Netflix is currently taking a more "industry-standard" approach to live streaming, focusing on stability and scalability, rather than optimizing encoding settings.
• Cloud gaming, which is essentially a specialized form of live streaming, also poses unique challenges due to the need for even lower latency.

2. How is Netflix adapting its encoding strategies to address the requirements of live streaming and cloud gaming?

• Netflix is having to disregard much of what it has learned over the past decade in optimizing encoding for on-demand video, as the approaches do not directly translate to live and cloud gaming.
• The company is currently prioritizing stability and scalability for its live streaming efforts, rather than fine-tuning encoding settings, as it continues to learn and adapt its strategies.
• Encoding for live and cloud gaming requires new approaches that cannot rely on the same techniques used for pre-recorded, on-demand content.